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Abstract 

The  main  aim  of  this  paper  is  to  develop  a prototype 
virtual  environment  for  training  Flight  Deck  Officers 
with  a view  to  study  the  types  of  interactions  required  in 
such  an  environment.  The  application  is  ideally  suited  to 
exploit  techniques  based  on  proprioception,  in  particular 
the  trainee’s  arm  signals. 

1.  Introduction 

A virtual  environment  is  a synthetic  sensory  experience 
that  communicates  physical  and  abstract  components  to 
a human  operator  or  participant  (Kalawsky,  1993). 
Virtual  Environments  (VE)  offer  greater  potential  to 
enhance  the  communication  between  the  human  and  the 
computer  as  they  offer  most  intuitive  and  natural 
interfaces.  They  have  been  exploited  in  diverse 
applications  ranging  from  medicine  to  training  soldiers. 
Their  potential  is  far  more  evident  in  training 
applications  (Nemire,  1998)  for  the  enriched  interaction 
styles  such  environments  support.  A typical  VE 
synthesises  one  or  more  sensory'  inputs  to  facilitate  a 
particular  user’s  task.  Exploiting  proprioception  (sensory 
awareness  of  parts  of  the  body)  enhances  interaction  in  a 
virtual  environment  (Mine,  1997).  One  form  of 
proprioception  is  the  use  of  body-relative  actions  called 
gestures  to  issue  commands  to  alter  the  environment. 
Current  research  work  in  this  area  includes  two-handed 
input  (Hand,  1997)  and  gesture-based  interaction  (Mapes 
& Moshell,  1995).  The  gestures  involved  in  the  present 
application  are  quite  unique,  and  thus  provide  an  ideal 
test  bed  for  exploring  3D  interactions  in  VE. 

Training  Flight  Deck  Officers  (FDO)  is  an  important 
aspect  of  Naval  operations  and  currently  uses  a range  of 
traditional  teaching  material  augmented  by  instructor- 
assisted  scenario  generation  (AIR  230  Course).  The 
instructor  directly  controls  the  scenario  presented  to  the 
trainee.  While  this  approach  has  some  strength,  we 
believe  that  a virtual  environment  offers  much 
significant  benefits  and  that  the  application  readily  lends 
itself  to  exploit  the  natural  interaction  styles,  such  as  arm 
signals,  that  are  inherent  in  the  training  of  Flight  Deck 
Operations  (Trott,  1999).  At  the  same  time,  the 
application  raises  several  challenges.  The  main  purpose 
of  this  article  is  to  present  the  results  of  our  initial 
prototype,  with  a view  to  enhance  the  model.  The 
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development  of  the  application  is  by  no  means  complete, 
and  should  be  treated  as  an  initial  investigation. 

The  paper  is  organised  as  follows.  A brief  motivation  is 
presented  in  Section  2.  The  problem  we  are  attempting  to 
address  in  this  paper  is  described  in  Section  3,  together 
with  typical  training  scenarios.  The  details  of  developing 
the  Virtual  Environment  are  given  in  Section  4.  Some  of 
the  points  highlighted  in  developing  the  prototype  are 
summarised  in  Section  5. 

2.  Rationale 

Exploiting  natural  interaction  metaphors  offered  by  the 
application  can  enhance  the  current  set  up  for  supporting 
the  training  of  Flight  Deck  Officers.  The  main 
motivation  for  the  present  study  can  be  summarised  as 
follows. 

• To  provide  an  enhanced  training  environment  for  the 
trainee. 

• To  allow  interactions  using  natural  metaphors  that 
will  enhance  the  experience  of  a flight  deck  officer  in 
which  he/she  will  be  able  to  control  the  environment 
in  response  to  his/her  actions.  For  example,  ask  the 
helicopter  to  move  to  the  next  gate  position  in 
response  to  an  arm  signal. 

3.  The  Problem 

Current  Practice 

Currently  the  British  Royal  Navy  Flight  Deck  Officers 
(FDO)  arc  trained  at  RNAS  Culdrosc,  Cornwall, 
England.  Their  training  makes  extensive  use  of  real 
simulation,  that  is  real  people  using  real  equipment.  In 
this  case  the  real  equipment  is  an  actual  helicopter, 
although  training  is  not  performed  on-board  ship. 

If  the  weather  conditions  restrict  aircraft  flights  or 
aircraft  are  unavailable,  then  the  training  makes  use  of  a 
virtual  simulator.  This  consists  of  three  large  projection 
screens  that  display  images  from  three  front  projectors 
driven  by  three  networked  PCs.  The  system  shows  the 
view  as  seen  from  the  landing  deck  of  a frigate.  The 
trainee  stands  in  front  of  the  screens  and  directs  the  flight 
of  the  simulated  helicopter  using  the  appropriate  signals. 
The  class  instructor  who  is  sitting  behind  the  trainee  flies 
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the  helicopter.  The  direction  of  view  of  the  system  is 
fixed  and  cannot  take  into  account  the  direction  of  view 
of  the  trainee.  The  current  system  also  has  limited 
graphics  capability  and  environmental  effects  such  as 
reduced  visibility,  fog  and  variable  sea-state  are  barely 
implemented. 

Although  the  virtual  simulator  is  available,  final 
examinations  must  be  passed  using  the  real  equipment. 
There  are  two  main  reasons  for  this:  the  virtual  simulator 
is  unable  to  replicate  the  feel  of  being  exposed  to  the 
prevailing  weather  conditions  nor  the  feel  of  the 
proximity  of  the  helicopter  as  it  lands. 


Figure  1:  Flight  Deck  Operations  Simulator 


Some  of  the  limitations  cited  above  can  be  addressed  by 
developing  a virtual  environment,  which  offers  far 
greater  potential.  To  explore  these  possibilities,  a subset 
of  training  scenarios  are  considered  for  the  initial  design, 
which  are  explained  in  the  next  section. 

Example  Scenarios 

Flight  Deck  Officer's  training  consists  of  a number  of 
scenarios  including  Landing  and  Takeoff  (varying  angles 
of  approach).  Rotors  Running  Refuel,  Helicopter  In- 
flight Refuel,  Weapons  Loading,  Personnel  Transfer,  and 
Helicopter  Shut  Down  and  Start  Up  using  either  a Sea 
King/Lynx.  All  these  scenarios  offer  a rich  variety  of  3D 
interactions  that  enhance  the  learning  and  training 
experience.  For  the  purposes  of  this  investigation,  we 
have  selected  two  scenarios  — Helicopter  landing  and 
take  off  under  varying  environmental  conditions. 

The  trainee  FDO  immersed  in  the  virtual  environment 
makes  an  assessment  of  the  wind  speed  and  direction 
(not  currently  implemented)  and  then  signals  the  aircraft 
when  he  is  ready  to  receive  it.  It  is  assumed  that  the 
aircraft  is  out  of  radar-controlled  approach  and  is  within 
the  visual  range  for  FDO  to  take  control.  On  receipt  of 
the  signal,  the  aircraft  moves  to  its  next  waypoint  or 
gateway.  A typical  approach  of  an  aircraft  to  the  ship's 
flight  deck  is  shown  in  Figure  2. 


Figure  2:  A typical  approach  of  an  aircraft.  Helicopter 
begins  approach  relative  angle  165°  from  ships  head 
(1);  Aircraft  reaches  gate  (2);  Aircraft  alongside  flight 
deck  directly  of  the  ‘bum’  line  (3);  Traverses  across 
flight  deck  maintaining  its  hover  (4);  aircraft  descends 
lo  flight  deck  (5). 


4.  Development  of  a Virtual  Environment 

The  virtual  environment  consists  of  a visual  model  of  the 
flight  deck  and  its  associated  dynamics,  a visual  model 
of  a helicopter  (Sea  King)  and  its  associated  dynamics, 
and  finally  a visual  representation  of  the  flight  deck 
officer  including  body  articulation  (limited  to  hands). 
For  the  training  purposes,  few  environmental  effects 
such  as  fog  and  night  time  are  also  included.  These  are 
discussed  in  detail  in  the  following  sections. 

Flight  Deck  Officer 

A simple  model  of  a mannequin  is  used  to  represent  the 
flight  deck  officer.  The  body  articulation  is  limited  to 
arms  only.  Currently  there  are  nearly  56  distinct  gestures 
used  by  the  FDO.  Of  these,  9 gestures  (See  Figures  3 and 
4)  which  are  directly  relevant  for  launch  and  recovery 
operations  are  chosen  for  the  prototype.  An  existing 
model  of  a man  from  the  Division  software  library  has 
been  modified  to  facilitate  the  emulation  of  arm  signals. 
The  animation  of  the  limbs  is  achieved  using  the 
keyframes  animation  technique  where  each  frame 
describes  a particular  state  of  the  object,  for  example  its 
position  and  orientation.  Each  hand  signal  is  stored  as  a 
key  frame  animation  sequence  and  was  stored  in  a 
separate  library. 

The  FDO  is  required  to  carry  lighted  wands  at  night  in 
order  to  make  his  signals  visible  to  the  pilot. 
Accordingly,  our  virtual  FDO  has  two  lighted  wands, 
which  come  into  effect  during  night 
time  training. 


1-3 


Helicopter  Approach 

A 3D  model  of  the  Sea  King  helicopter  is  modified  to 
provide  realistic  rotor  disc  motion  and  the  addition  of 
navigation  lights  to  the  helicopter  stub  wings.  This  w'as 
achieved  by  mounting  a spotlight  just  in  front  of  the 
navigation  light  and  setting  an  appropriate  object 
luminescence  property  in  the  object’s  texture  file. 

The  movement  of  the  helicopter  is  governed  by  a series 
of  keyframe  sequences  in  a required  direction.  The 
helicopter  movement  is  triggered  by  an  event  (such  as 
receive,  approach,  move  away,  left,  right,  up,  down, 
hold,  wave  off)  raised  by  FDOs  hand  signal.  In  response 
to  this  signal,  the  aircraft  will  move  to  the  next  ‘gate’ 
position.  Once  the  aircraft  has  been  successfully  directed 
over  the  deck,  a final  signal  to  descend  to  the  deck  is 
given.  When  the  collision  between  the  helicopter  and  the 
deck  is  detected,  the  helicopter  object  is  parented  with 
the  deck  object,  so  that  the  helicopter  moves  in 
accordance  with  the  motion  of  the  flight  deck  and  that  of 
the  ship. 

Platform  Dynamics 

The  platform  consists  of  the  deck,  harpoon  grid  and  a 
model  of  the  FDO  standing  on  the  deck.  To  keep  the 
frame  rates  to  a minimum,  a simple  animation  sequence 
is  created  for  the  platform  that  emulates  the  motion  of  a 
ship. 

Environmental  Effects 

Environmental  effects  such  as  lighting  conditions  and 
visibility  effects  can  be  easily  incorporated  into  the 
virtual  environment.  To  effectively  light  the  scene  and 
allow  for  a number  of  different  lighting  conditions  five 
light  sources  are  used.  Four  of  these  light  sources  are 
used  for  scenery  lighting  and  the  remaining  light  is  used 
to  illuminate  the  deck.  An  appropriate  texture  is  applied 
to  the  sky  to  produce  an  impression  of  a marginally 
cloudy  day.  Fog  is  emulated  using  the  library  function 
dvFog  which  allows  a colour  and  distance  parameter 
(beyond  which  the  objects  are  invisible)  to  be  specified. 
Note  that  when  fog  is  enabled,  the  sky  is  obscured;  and 
affects  the  intervisibility  computations. 

Virtual  Command  Interface 

The  prototype  environment  is  developed  using  the 
Division  software  dVS/dVISE.  Due  to  the  current 
limitations,  the  arm  signals  of  the  trainee  are  simulated 
using  virtual  menus  (See  Figure  5).  A limited  number  of 
training  manoeuvres  is  implemented  for  helicopter 
landing  and  takeoff  under  different  environmental 
conditions.  The  use  of  directional  sound  is  also  explored 
with  limited  success.  Note  that  for  a fully  functional 
immersive  environment,  appropriate  hardware,  and 
additional  software  for  gesture  analysis  should  replace 
the  virtual  menus  mentioned  above. 

5.  Remarks 

As  the  main  focus  in  this  study  on  developing  a virtual 
environment  for  training,  no  specific  experiments  were 


conducted  to  evaluate  the  overall  benefit  of  such  a 
system.  However,  the  exercise  has  revealed  several 
important  factors. 

Top  most  in  the  list  is  the  need  for  a software  component 
for  gesture  interpretation.  The  computational  demands 
for  training  purposes  are  moderate,  as  the  objects  in  the 
virtual  environment  remain  fairly  static.  Selection  of 
items  using  Virtual  menus  is  not  intuitive,  and  could  be 
enhanced  using  additional  visual  cues. 

The  next  stage  of  the  work  is  an  investigation  into  the 
recognition  of  various  arm  signals  using  a single  tracking 
device  in  each  hand.  Simply  knowing  where  each  hand 
and  it’s  orientation  is  insufficient.  It  is  expected  that 
knowledge  of  how  each  hand  has  recently  moved  will  be 
required  to  determine  the  relevant  signal.  For  example, 
arm  signals  for  FDO  Up  and  FDO  Down  (See  Figure  4) 
trace  the  same  path,  but  differ  in  start  and  finish 
positions.  We  envisage  that  it  will  not  be  necessary  to 
have  tracking  devices  at  either  the  elbows  or  shoulders. 
The  use  of  a neural  network  to  facilitate  this  task  is 
expected. 

6.  Conclusions 

The  prototype  environment  for  FDO  training  has 
highlighted  some  of  the  requirements  that  are  essential 
for  a fully  immersive  tool.  There  is  a clear  need  to  track 
the  position  of  both  head  and  two  arms.  While  the 
current  tracking  system  is  capable  of  tracking  position 
data  up  to  four  trackers,  this  is  not  currently 
implemented,  and  will  be  pursued  in  a future 
investigation.  Use  of  directional  audio  cues  have  been 
explored  very  briefly,  but  needs  a detailed  study. 
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FDO  Ready  to  receive  - you  are  cleared  to  land 


FDO  Approach 


FDO  Move  left 


FDO  Move  right 


Figure  3:  A subset  of  Flight  Deck  Officer’s  hand  signals 


FDO  Down 


FDO  Wave-Off 


FDO  Hold 


Figure  4:  A subset  of  Flight  Deck  Officer’s  arm  signals  (ctd.) 
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Figure  5:  Virtual  menues  used  in  the  virtual  environment 


